{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Introduction\n",
    "\n",
    "This example demonstrates how to convert a network from [Caffe's Model Zoo](https://github.com/BVLC/caffe/wiki/Model-Zoo) for use with Lasagne. We will be using the Lenet trained for MNIST.\n",
    "\n",
    "We will create a set of Lasagne layers corresponding to the Caffe model specification (prototxt), then copy the parameters from the caffemodel file into our model."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "# Converting from Caffe to Lasagne\n",
    "### Download the required files\n",
    "\n",
    "First we download `cifar10_nin.caffemodel` and `model.prototxt`. The supplied `train_val.prototxt` was modified to replace the data layers with an input specification, and remove the unneeded loss/accuracy layers."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Import Caffe\n",
    "\n",
    "To load the saved parameters, we'll need to have Caffe's Python bindings installed."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import sys\n",
    "caffe_root = '/home/xilinx/caffe/'  # this file should be run from {caffe_root}/examples (otherwise change this line)\n",
    "sys.path.insert(0, caffe_root + 'python')\n",
    "import caffe"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Load the pretrained Caffe network"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "net_caffe = caffe.Net('lenet.prototxt', 'lenet_iter_10000.caffemodel', caffe.TEST)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Import Lasagne"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import lasagne\n",
    "from lasagne.layers import InputLayer, DropoutLayer, DenseLayer, NonlinearityLayer\n",
    "#from lasagne.layers.dnn import Conv2DDNNLayer as ConvLayer\n",
    "#from lasagne.layers import Conv2DLayer as ConvLayer\n",
    "from lasagne.layers import Pool2DLayer as PoolLayer\n",
    "from lasagne.nonlinearities import softmax, rectify, linear\n",
    "import conv_fpga\n",
    "from conv_fpga import FPGA_LENET\n",
    "from conv_fpga import FPGAQuickTest\n",
    "from conv_fpga import Conv2DLayer as ConvLayer\n",
    "from conv_fpga import FPGAWeightLoader as FPGALoadW\n",
    "from lasagne.utils import floatX"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Create a Lasagne network\n",
    "Layer names match those in `model.prototxt`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "net = {}\n",
    "net['input'] = InputLayer((None, 1, 28, 28))\n",
    "net['conv1'] = ConvLayer(net['input'], num_filters=20, filter_size=5, nonlinearity=linear)\n",
    "net['pool1'] = PoolLayer(net['conv1'], pool_size=2, stride=2, mode='max', ignore_border=False)\n",
    "net['conv2'] = ConvLayer(net['pool1'], num_filters=50, filter_size=5, nonlinearity=linear)\n",
    "net['pool2'] = PoolLayer(net['conv2'], pool_size=2, stride=2, mode='max', ignore_border=False)\n",
    "net['ip1'] = DenseLayer(net['pool2'], num_units=500, nonlinearity = rectify)\n",
    "#net['relu1'] =  NonlinearityLayer(net['ip1'], rectify)\n",
    "net['ip2'] = DenseLayer(net['ip1'], num_units=10, nonlinearity = None)\n",
    "net['prob'] = NonlinearityLayer(net['ip2'], softmax)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "### Copy the parameters from Caffe to Lasagne"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "\n",
    "layers_caffe = dict(zip(list(net_caffe._layer_names), net_caffe.layers))\n",
    "\n",
    "for name, layer in net.items():\n",
    "    try:\n",
    "#        layer.W.set_value(layers_caffe[name].blobs[0].data)\n",
    "#        layer.b.set_value(layers_caffe[name].blobs[1].data)     \n",
    "        if name=='ip1'or name=='ip2':\n",
    "            layer.W.set_value(np.transpose(layers_caffe[name].blobs[0].data))\n",
    "            layer.b.set_value(layers_caffe[name].blobs[1].data)\n",
    "        else:\n",
    "            layer.W.set_value(layers_caffe[name].blobs[0].data[:,:,::-1,::-1])\n",
    "            layer.b.set_value(layers_caffe[name].blobs[1].data)\n",
    "            #print((layers_caffe[name].blobs[1].data))\n",
    "            \n",
    "    except AttributeError:\n",
    "        continue"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Copy the parameters from CPU to FPGA OnChip Memory"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "weight shape (20, 1, 5, 5)\n",
      "kermax 0.549957931042\n",
      "kermin -0.610449552536\n",
      "Elapsed Test Time:  0.0021906859999987205\n",
      "weight shape (50, 20, 5, 5)\n",
      "kermax 0.238257542253\n",
      "kermin -0.16036234796\n",
      "Elapsed Test Time:  0.0007058549999996444\n",
      "weight shape (500, 50, 4, 4)\n",
      "kermax 0.105541154742\n",
      "kermin -0.101853817701\n",
      "Elapsed Test Time:  0.009190980999996157\n",
      "weight shape (10, 500, 1, 1)\n",
      "kermax 0.271254092455\n",
      "kermin -0.267873495817\n",
      "Elapsed Test Time:  0.000254356999995764\n"
     ]
    }
   ],
   "source": [
    "#FPGALoadW(weight, status, IFDim, OFDim, PadDim)\n",
    "weight = net['conv1'].W.get_value()\n",
    "FPGALoadW(weight, 1, 28, 24, 0)\n",
    "weight = net['conv2'].W.get_value()\n",
    "FPGALoadW(weight, 2, 12, 8, 0)\n",
    "weight = net['ip1'].W.get_value()\n",
    "weight = np.transpose(weight)\n",
    "weight = weight.reshape(500, 50, 4, 4)\n",
    "FPGALoadW(weight, 3, 4, 1, 0, flip_filters=False)\n",
    "weight = net['ip2'].W.get_value()\n",
    "weight = np.transpose(weight)\n",
    "weight = weight.reshape(10, 500, 1, 1)\n",
    "FPGALoadW(weight, 4, 1, 1, 0, flip_filters=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "# Trying it out\n",
    "Let's see if that worked. \n",
    "\n",
    "### Import numpy and set up plotting\n",
    "### Import time"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import gzip\n",
    "import _pickle as cPickle\n",
    "import matplotlib.pyplot as plt\n",
    "import time\n",
    "\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Download some test data\n",
    "Load test mnist handwritting data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.image.AxesImage at 0x2a12fe10>"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAATYAAAEzCAYAAAC7cS8aAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAADytJREFUeJzt3W2MXOV5xvHrIm4/BKvIRbGt2A1TFJVWaasVoUiR+2Ej\n0oCqRiaoocRJZEhLLRW3qKQqDo7qdcRKpk3c0jRUKhhinNA00BA7X3iJ6DoiVWI3YXkJ5kVKxoFg\nr62KRHXVSqS++2EOMDYz7LMzZ/bs3P7/pBGzZ+49z318zOXnzMwz44gQAGRyVtMNAEDdCDYA6RBs\nANIh2ACkQ7ABSIdgA5DOsmF+2fZlkv5OnYDcFRG39Kjh/SQARiIi3Gu7B30fm+2zJD0n6RJJL0k6\nKOmqiHjmtLqQtp322zOSJgcad2mZEcexVMxo/I9B4jgWYnvfYBvmUvRiSc9HxOGIeEXSlyWtH2J/\nAFCLYYJtjaQXun5+sdoGAI1q6MWDVjPD1q7VdAM1aTXdQA1aTTdQk1bTDdSk1ejow7x48GNJ7+j6\neW21rYeZrvstNX3Q9Wk13UBNWk03UINW0w3UpNV0AzVpjWCf7eo2v2GC7aCkd9o+T9IRSVdJ+nDv\n0skhhgEA6Y2Tov19KwcOtoj4P9ubJT2k19/ucWjQ/QFAXYZ6H1tEPCDpgpp6AYBasPIAQDoEG4B0\nCDYA6RBsANIh2ACkQ7ABSIdgA5AOwQYgHYINQDoEG4B0CDYA6RBsANIh2ACkQ7ABSIdgA5AOwQYg\nHYINQDoEG4B0CDYA6RBsANIh2ACkQ7ABSIdgA5AOwQYgHYINQDoEG4B0CDYA6RBsANIh2ACkQ7AB\nSIdgA5AOwQYgHYINQDoEG4B0CDYA6RBsANIh2ACkQ7ABSIdgA5AOwQYgHYINQDrLhvll221JP5V0\nUtIrEXFxHU0BwDCGCjZ1Am0yIl6uoxkAqMOwl6KuYR8AUKthQykkPWz7oO1r62gIAIY17KXouog4\nYvtt6gTcoYh4tI7GAGBQQwVbRByp/nvc9v2SLpbUI9hmuu63qhsALES7us1v4GCz/VZJZ0XECdtn\nS3q/pO29qycHHQYAKi2dOina37dymBnbKkn3245qP1+KiIeG2B8A1GLgYIuIH0qaqLEXAKgFb9UA\nkA7BBiAdgg1AOgQbgHQINgDpEGwA0iHYAKRDsAFIh2ADkA7BBiCdYT+2CKebnSoqix+4qO7wB99W\nVHdEby+qe8+fzxbV6QtlZfrJVGEhsHiYsQFIh2ADkA7BBiAdgg1AOgQbgHQINgDpEGwA0iHYAKRD\nsAFIh5UHNYsPla0omH6+dI/Ha627WWX9rSscdfKvCgvR26ayMl8XZYVfmxq4lUyYsQFIh2ADkA7B\nBiAdgg1AOgQbgHQINgDpEGwA0iHYAKRDsAFIh5UHNfOmsneIr//EPxfV7d394bL9bSzb3536eFHd\ninP/t6hu+tNFZdpa9pUMmn6prK5uv1hYt/Hssrq//e/CHRb++e2J3y+q+5h/vXDg3JixAUiHYAOQ\nDsEGIB2CDUA6BBuAdAg2AOkQbADSIdgApEOwAUjHEYWfpT7oAHZI20Y6BkbhA2VlO95dVrdlpqzu\nM5NldXVrF9b9ww+Lyr6v84vq7i8cdl/MFNUd8L8V7jGD7YqInl/iMe+MzfYu23O2n+jatsL2Q7af\ntf2g7XPqbBcAhlFyKXqXpEtP27ZF0jci4gJJj0j6ZN2NAcCg5g22iHhU0sunbV4vaXd1f7eky2vu\nCwAGNuiLBysjYk6SIuKopJX1tQQAw6nrY4vmeQViput+q7oBwEK0Vfoqz6DBNmd7VUTM2V4t6dib\nl08OOAwAvKqlUydF+/tWll6Kurq9ap+kq6v7GyXtLW0NAEat5O0e90j6d0m/YvtHtq+RtEPS79h+\nVtIl1c8AsCTMeykaERv6PPS+mnsBgFqw8gBYiP+YKirbcVHPN8S/wY0fKhvW9z5eVqivFtZlMMTK\nAwAYNwQbgHQINgDpEGwA0iHYAKRDsAFIh2ADkA7BBiAdgg1AOnV9bBEw5v6oqOqFwhUFdxSO6hsL\nV/7cO1W4R0jM2AAkRLABSIdgA5AOwQYgHYINQDoEG4B0CDYA6RBsANIh2ACkw8oDQNJUfK6obnfZ\nwgN9oHDc7WULHrBAzNgApEOwAUiHYAOQDsEGIB2CDUA6BBuAdAg2AOkQbADSIdgApMPKA+S2eaqo\n7O0uW1JwrHDYi7YUfpfBjqnCPWIhmLEBSIdgA5AOwQYgHYINQDoEG4B0CDYA6RBsANIh2ACkQ7AB\nSMcRhe+QHnQAO6RtIx0D6OezUbZW4H/8j0V1W+8pG9cbbisr1FxhHd5ouyKi55KReWdstnfZnrP9\nRNe2bbZftP296nZZne0CwDBKLkXvknRpj+07I+LC6vZAzX0BwMDmDbaIeFTSyz0eKvwiMgBYXMO8\neLDZ9qztO2yfU1tHADCkQT+26DZJn46IsH2zpJ2S/rB/+UzX/VZ1A4CFaFe3+Q0UbBFxvOvH2yV9\n/c1/Y3KQYQCgS0unTor2960svRS1up5Ts72667ErJD1V3BsAjNi8Mzbb96gz5TrX9o/UeVPae21P\nSDqpztxw0wh7BIAFmTfYImJDj813jaAXAKgF33mAMfVrRVU3XLu9qO4LhaP6m6UrdaYK6zAKrBUF\nkA7BBiAdgg1AOgQbgHQINgDpEGwA0iHYAKRDsAFIh2ADkA4rDzCW/jKeKaqbLvw41K0PltVdc+lU\nWSEaxYwNQDoEG4B0CDYA6RBsANIh2ACkQ7ABSIdgA5AOwQYgHYINQDqsPMDSct9UUdkaly0puLBw\nWN/LdxlkwowNQDoEG4B0CDYA6RBsANIh2ACkQ7ABSIdgA5AOwQYgHYINQDqsPMAiubKoKj5VtqJg\nunDUi04UFi6fKizEOGDGBiAdgg1AOgQbgHQINgDpEGwA0iHYAKRDsAFIh2ADkA7BBiCdeVce2F4r\n6W5JqySdlHR7RPy97RWS/kXSeZLakq6MiJ+OsFcsSb9QVBXXvauobvrzZaNunSir82q+y+BMVDJj\n+5mkGyLiXZLeI+k6278qaYukb0TEBZIekfTJ0bUJAOXmDbaIOBoRs9X9E5IOSVorab2k3VXZbkmX\nj6pJAFiIBT3HZrslaULStyWtiog5qRN+klbW3RwADKI42Gwvl3SfpOurmdvpT16UPpkBACNV9LFF\ntpepE2p7ImJvtXnO9qqImLO9WtKx/nuY6brfqm4AsBDt6ja/0s9ju1PS0xFxa9e2fZKulnSLpI2S\n9vb4vcpk4TAA0E9Lp06K9vetLHm7xzpJH5H0pO3H1LnkvEmdQPuK7Y9LOqzSTxIEgBGbN9gi4luS\n3tLn4ffV2w4ADI+VBwDS4TsPMJzVNxSVTX/+E7UO65sLX4T/valax8V4YMYGIB2CDUA6BBuAdAg2\nAOkQbADSIdgApEOwAUiHYAOQDsEGIB1WHqCPqaKq+DkX1U0XjvpY7CkrdOkecSZixgYgHYINQDoE\nG4B0CDYA6RBsANIh2ACkQ7ABSIdgA5AOwQYgHVYeoKfPxp8U1U2XLTwo9q8f/Ghh5VS9AyMVZmwA\n0iHYAKRDsAFIh2ADkA7BBiAdgg1AOgQbgHQINgDpEGwA0mHlwZnmL6aKyn7DZUsKDgzRCjAqzNgA\npEOwAUiHYAOQDsEGIB2CDUA6BBuAdAg2AOkQbADSIdgApDPvygPbayXdLWmVpJOS/ikiPmd7m6Rr\nJR2rSm+KiAdG1ilq8cd/c2tR3YHP1Dvu1omyuk/N1jsuzkwlS6p+JumGiJi1vVzSd20/XD22MyJ2\njq49AFi4eYMtIo5KOlrdP2H7kKQ11cM1f0cRAAxvQc+x2W5JmpD0nWrTZtuztu+wfU7NvQHAQIqD\nrboMvU/S9RFxQtJtks6PiAl1ZnRckgJYEoo+tsj2MnVCbU9E7JWkiDjeVXK7pK/338NM1/1WdQOA\nhWhXt/mVfh7bnZKejojXXlKzvbp6/k2SrpD0VP9fnywcBgD6aenUSdH+vpUlb/dYJ+kjkp60/Zik\nkHSTpA22J9R5C0hb0qZB2wWAOpW8KvotSW/p8RDvWQOwJLHyAEA6fOcBhrL1o2V1/uLjhXucGrQV\n4DXM2ACkQ7ABSIdgA5AOwQYgHYINQDoEG4B0CDYA6RBsANIh2ACk44gY7QB2SNtGOgaAM9F2RUTP\nT/FmxgYgHYINQDoEG4B0CDYA6RBsANJpKNjazQxbu3bTDdSk3XQDNWg33UBN2k03UJN2o6MTbENp\nN91ATdpNN1CDdtMN1KTddAM1aTc6OpeiANIh2ACks0grDwCgfv1WHow82ABgsXEpCiAdgg1AOosa\nbLYvs/2M7eds37iYY9fJdtv247Yfs32g6X5K2d5le872E13bVth+yPazth+0fU6TPZbocxzbbL9o\n+3vV7bImeyxhe63tR2x/3/aTtv+s2j4256THMfxptb3R87Foz7HZPkvSc5IukfSSpIOSroqIZxal\ngRrZ/oGkd0fEy033shC2f1vSCUl3R8RvVttukfSfEfHX1T82KyJiS5N9zqfPcWyT9F8RsbPR5hbA\n9mpJqyNi1vZySd+VtF7SNRqTc/Imx/AHavB8LOaM7WJJz0fE4Yh4RdKX1fkDGEfWGF7GR8Sjkk4P\n4/WSdlf3d0u6fFGbGkCf45A652VsRMTRiJit7p+QdEjSWo3ROelzDGuqhxs7H4v5P+caSS90/fyi\nXv8DGDch6WHbB21f23QzQ1oZEXNS5y+ppJUN9zOMzbZnbd+xlC/ferHdkjQh6duSVo3jOek6hu9U\nmxo7H2M361gi1kXEhZJ+V9J11aVRFuP6/p/bJJ0fEROSjkoap0vS5ZLuk3R9Nes5/Rws+XPS4xga\nPR+LGWw/lvSOrp/XVtvGTkQcqf57XNL96lxmj6s526uk154vOdZwPwOJiOPx+hPGt0v6rSb7KWV7\nmTqBsCci9labx+qc9DqGps/HYgbbQUnvtH2e7Z+XdJWkfYs4fi1sv7X610m2z5b0fklPNdvVglin\nPvexT9LV1f2Nkvae/gtL1CnHUQXAq67Q+JyTOyU9HRG3dm0bt3PyhmNo+nws6sqD6iXfW9UJ1F0R\nsWPRBq+J7V9WZ5YWkpZJ+tK4HIfteyRNSjpX0pw637LzNUn3SvolSYclXRkRP2mqxxJ9juO96jy/\nc1Kdj5bY9OrzVEuV7XWSvinpSXX+PoWkmyQdkPQVjcE5eZNj2KAGzwdLqgCkw4sHANIh2ACkQ7AB\nSIdgA5AOwQYgHYINQDoEG4B0CDYA6fw/RP1owt05gqAAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x2a47ed70>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "data = np.load('test_cpr.npz')\n",
    "test_data = data['data'].reshape(10000, 1, 28, 28)\n",
    "#test_data = test_data.transpose(0, 1, 3, 2)\n",
    "test_label = data['label']\n",
    "##\n",
    "plt.figure(figsize=(5, 5))\n",
    "plt.imshow(test_data[0][0], interpolation='nearest', cmap=plt.get_cmap('gray'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### FPGA Deployment (Lasagne Layer)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "FPGA_net = {}\n",
    "FPGA_net['input'] = InputLayer((None, 1, 28, 28))\n",
    "FPGA_net['lenet'] = FPGA_LENET(FPGA_net['input'])\n",
    "FPGA_net['prob'] = NonlinearityLayer(FPGA_net['lenet'], softmax)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Elapsed Test Time:  1.1257071940000003\n",
      "CPU times: user 2.42 s, sys: 70 ms, total: 2.49 s\n",
      "Wall time: 2.5 s\n"
     ]
    }
   ],
   "source": [
    "batch_size = 600\n",
    "\n",
    "%time prob = lasagne.layers.get_output(FPGA_net['prob'], floatX(test_data[0:batch_size]), deterministic=True).eval()\n",
    "FPGA_predicted = np.argmax(prob, 1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.983333333333\n"
     ]
    }
   ],
   "source": [
    "FPGA_accuracy = np.mean(FPGA_predicted == test_label[0][0:batch_size])\n",
    "#print(FPGA_predicted)\n",
    "#print(test_label[0][0:600])\n",
    "print(FPGA_accuracy)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### FPGA Deployment (QuickTest Function)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 88,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Elapsed Test Time:  7.510732433999692\n"
     ]
    },
    {
     "ename": "TimeoutError",
     "evalue": "DMA wait timed out.",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mTimeoutError\u001b[0m                              Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-88-9bde0c3c5166>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[1;32m      3\u001b[0m \u001b[0mOFMDim\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      4\u001b[0m \u001b[0mOFMCH\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;36m10\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 5\u001b[0;31m \u001b[0mget_ipython\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mmagic\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'time FPGA_output = FPGAQuickTest(test_data[8001:], batch_size, OFMDim, OFMCH)'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m      6\u001b[0m \u001b[0mFPGA_predicted\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mnp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0margmax\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mFPGA_output\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mreshape\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mbatch_size\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m-\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      7\u001b[0m \u001b[0;31m#np.save('FPGA_output', FPGA_output)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/usr/local/lib/python3.4/dist-packages/IPython/core/interactiveshell.py\u001b[0m in \u001b[0;36mmagic\u001b[0;34m(self, arg_s)\u001b[0m\n\u001b[1;32m   2144\u001b[0m         \u001b[0mmagic_name\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0m_\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mmagic_arg_s\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0marg_s\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mpartition\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m' '\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   2145\u001b[0m         \u001b[0mmagic_name\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mmagic_name\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mlstrip\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mprefilter\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mESC_MAGIC\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 2146\u001b[0;31m         \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mrun_line_magic\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mmagic_name\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mmagic_arg_s\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   2147\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   2148\u001b[0m     \u001b[0;31m#-------------------------------------------------------------------------\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/usr/local/lib/python3.4/dist-packages/IPython/core/interactiveshell.py\u001b[0m in \u001b[0;36mrun_line_magic\u001b[0;34m(self, magic_name, line)\u001b[0m\n\u001b[1;32m   2065\u001b[0m                 \u001b[0mkwargs\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m'local_ns'\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0msys\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_getframe\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mstack_depth\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mf_locals\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   2066\u001b[0m             \u001b[0;32mwith\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mbuiltin_trap\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 2067\u001b[0;31m                 \u001b[0mresult\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mfn\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   2068\u001b[0m             \u001b[0;32mreturn\u001b[0m \u001b[0mresult\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   2069\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m<decorator-gen-59>\u001b[0m in \u001b[0;36mtime\u001b[0;34m(self, line, cell, local_ns)\u001b[0m\n",
      "\u001b[0;32m/usr/local/lib/python3.4/dist-packages/IPython/core/magic.py\u001b[0m in \u001b[0;36m<lambda>\u001b[0;34m(f, *a, **k)\u001b[0m\n\u001b[1;32m    186\u001b[0m     \u001b[0;31m# but it's overkill for just that one bit of state.\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    187\u001b[0m     \u001b[0;32mdef\u001b[0m \u001b[0mmagic_deco\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0marg\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 188\u001b[0;31m         \u001b[0mcall\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;32mlambda\u001b[0m \u001b[0mf\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m*\u001b[0m\u001b[0ma\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mk\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0mf\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0ma\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mk\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    189\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    190\u001b[0m         \u001b[0;32mif\u001b[0m \u001b[0mcallable\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0marg\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/usr/local/lib/python3.4/dist-packages/IPython/core/magics/execution.py\u001b[0m in \u001b[0;36mtime\u001b[0;34m(self, line, cell, local_ns)\u001b[0m\n\u001b[1;32m   1174\u001b[0m         \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1175\u001b[0m             \u001b[0mst\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mclock2\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1176\u001b[0;31m             \u001b[0mexec\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mcode\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mglob\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mlocal_ns\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   1177\u001b[0m             \u001b[0mend\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mclock2\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1178\u001b[0m             \u001b[0mout\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m<timed exec>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n",
      "\u001b[0;32m/home/xilinx/jupyter_notebooks/PYNQ_CNN/Theano/Lenet/conv_fpga.py\u001b[0m in \u001b[0;36mFPGAQuickTest\u001b[0;34m(test_data, batch_size, OFMDim, OFMCH)\u001b[0m\n\u001b[1;32m    294\u001b[0m     \u001b[0minput_val\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mnp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mappend\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mbatch_size\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m5\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m28\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m10\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m2\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0minput_val\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    295\u001b[0m     \u001b[0minput_val\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0minput_val\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mastype\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mint\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 296\u001b[0;31m     \u001b[0moutput_fpga\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mdouble\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0minput_val\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mhw_value\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    297\u001b[0m     \u001b[0moutput_mat_tmp\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0moutput_fpga\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m8\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mreshape\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mbatch_size\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mOFMDim\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0mOFMDim\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mOFMCH\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    298\u001b[0m     \u001b[0moutput_mat_tmp\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0moutput_mat_tmp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mtranspose\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m2\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/home/xilinx/jupyter_notebooks/PYNQ_CNN/Theano/Lenet/conv_fpga.py\u001b[0m in \u001b[0;36mhw_value\u001b[0;34m(self)\u001b[0m\n\u001b[1;32m     38\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     39\u001b[0m   \u001b[0;32mdef\u001b[0m \u001b[0mhw_value\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 40\u001b[0;31m     \u001b[0;32mreturn\u001b[0m \u001b[0mexecute_hardware\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     41\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     42\u001b[0m   \u001b[0;32mdef\u001b[0m \u001b[0m__str__\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/home/xilinx/jupyter_notebooks/PYNQ_CNN/Theano/Lenet/conv_fpga.py\u001b[0m in \u001b[0;36mexecute_hardware\u001b[0;34m(plan)\u001b[0m\n\u001b[1;32m    181\u001b[0m   \u001b[0mend_time\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mtime\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mprocess_time\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    182\u001b[0m   \u001b[0mprint\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"Elapsed Test Time: \"\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mend_time\u001b[0m\u001b[0;34m-\u001b[0m\u001b[0mstart_time\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 183\u001b[0;31m   \u001b[0mret_dma\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mwait\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    184\u001b[0m   \u001b[0mbytes_read\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mret_dma_mmio\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mread\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;36m0x58\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    185\u001b[0m   \u001b[0mffi\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mpynq\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mdrivers\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mdma\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mffi\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/usr/local/lib/python3.4/dist-packages/pynq/drivers/dma.py\u001b[0m in \u001b[0;36mwait\u001b[0;34m(self, wait_timeout)\u001b[0m\n\u001b[1;32m    439\u001b[0m         \u001b[0;32mwith\u001b[0m \u001b[0mtimeout\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mseconds\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mwait_timeout\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0merror_message\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mError\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    440\u001b[0m             \u001b[0;32mwhile\u001b[0m \u001b[0;32mTrue\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 441\u001b[0;31m                 \u001b[0;32mif\u001b[0m \u001b[0mlibdma\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mXAxiDma_Busy\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mDMAengine\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mdirection\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m==\u001b[0m \u001b[0;36m0\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    442\u001b[0m                     \u001b[0;32mbreak\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    443\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/usr/local/lib/python3.4/dist-packages/pynq/drivers/dma.py\u001b[0m in \u001b[0;36mhandle_timeout\u001b[0;34m(self, signum, frame)\u001b[0m\n\u001b[1;32m    173\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    174\u001b[0m     \u001b[0;32mdef\u001b[0m \u001b[0mhandle_timeout\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0msignum\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mframe\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 175\u001b[0;31m         \u001b[0;32mraise\u001b[0m \u001b[0mTimeoutError\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0merror_message\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    176\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    177\u001b[0m     \u001b[0;32mdef\u001b[0m \u001b[0m__enter__\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mTimeoutError\u001b[0m: DMA wait timed out."
     ]
    }
   ],
   "source": [
    "batch_size = 2000\n",
    "\n",
    "OFMDim = 1\n",
    "OFMCH = 10\n",
    "%time FPGA_output = FPGAQuickTest(test_data[8000:], batch_size, OFMDim, OFMCH)\n",
    "FPGA_predicted = np.argmax(FPGA_output.reshape(batch_size, -1), 1)\n",
    "#np.save('FPGA_output', FPGA_output)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 87,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.994\n"
     ]
    }
   ],
   "source": [
    "FPGA_accuracy = np.mean(FPGA_predicted == test_label[0][6000:6000+batch_size])\n",
    "print(FPGA_accuracy)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Testbench: Check output for each layer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "upper bound 23.2141457003\n",
      "lower bound -10.4159099288\n",
      "[[ -2.70613421  -0.90336153  -0.06154513   3.81282911  -3.91745218\n",
      "   -2.2983122  -10.41590993  15.12046854  -0.12228002   1.31426107]\n",
      " [  4.86611141   4.82199188  23.2141457   -1.51167638  -8.5911133\n",
      "   -9.40902512   0.47449177  -3.77196135   0.23502052  -9.25238383]\n",
      " [ -2.31401035   8.62488298   0.25990902  -5.28808693   0.89774727\n",
      "   -2.1109134   -4.00580258   0.76420222   2.76597867  -3.25794247]\n",
      " [ 12.48768544  -5.47143732   1.46074695  -6.13742732  -2.10245264\n",
      "   -2.91053723   2.5690011    0.65909134  -3.22218605   1.29327217]\n",
      " [ -2.20250484  -5.73974968  -2.03936998  -2.61254005  13.50087001\n",
      "   -4.67510671  -2.59636619  -0.8826095    1.86436696   7.05478231]]\n"
     ]
    }
   ],
   "source": [
    "conv_output = lasagne.layers.get_output(net['ip2'], floatX(test_data[0:5]), deterministic=True).eval()\n",
    "print('upper bound', np.amax(conv_output))\n",
    "print('lower bound', np.amin(conv_output))\n",
    "print(conv_output)\n",
    "np.allclose(FPGA_output, conv_output)\n",
    "np.save('cpu_output', conv_output)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "A [[[-1.44444444]]\n",
      "\n",
      " [[-4.11111111]]\n",
      "\n",
      " [[-1.05555556]]\n",
      "\n",
      " [[-1.5       ]]\n",
      "\n",
      " [[ 9.5       ]]\n",
      "\n",
      " [[-3.33333333]]\n",
      "\n",
      " [[-1.83333333]]\n",
      "\n",
      " [[-1.        ]]\n",
      "\n",
      " [[ 0.94444444]]\n",
      "\n",
      " [[ 5.16666667]]]\n",
      "B [ -2.20250484  -5.73974968  -2.03936998  -2.61254005  13.50087001\n",
      "  -4.67510671  -2.59636619  -0.8826095    1.86436696   7.05478231]\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "False"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "A = np.load('FPGA_output.npy')\n",
    "B = np.load('cpu_output.npy')\n",
    "print('A',A[4])\n",
    "print('B',B[4])\n",
    "np.allclose(A,B)\n",
    "#print(np.amax(B))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "weight shape (20, 1, 5, 5)\n",
      "input shape (1, 1, 28, 28)\n",
      "[[[[ -1.23521651e+00  -2.04537287e+00  -6.32075214e-01 ...,\n",
      "     -1.63300213e+00  -2.55450387e+00  -2.07054086e+00]\n",
      "   [ -1.40137811e+00  -1.77353978e+00  -1.84130565e+00 ...,\n",
      "     -1.33434864e+00  -2.22999067e+00  -2.42328846e+00]\n",
      "   [ -1.26774366e+00  -1.71084404e+00  -1.31209231e+00 ...,\n",
      "     -2.24487298e+00  -2.73626462e+00  -1.29489929e+00]\n",
      "   ..., \n",
      "   [ -1.62035386e+00  -7.08239809e-01  -2.27327630e+00 ...,\n",
      "     -1.44700859e+00  -3.67266482e+00  -3.81733384e+00]\n",
      "   [ -2.17358027e+00  -1.13402332e+00  -3.92598738e-03 ...,\n",
      "     -2.06156307e+00  -1.44563193e+00  -2.10430690e+00]\n",
      "   [ -1.27752032e+00  -1.06594470e+00  -1.12218224e+00 ...,\n",
      "     -1.30819874e-01  -1.01351195e+00  -1.54660986e+00]]\n",
      "\n",
      "  [[ -2.60145689e-01  -5.13031884e-02   1.01291686e+00 ...,\n",
      "      8.97090336e-01   6.42144868e-01   1.94622497e-01]\n",
      "   [ -1.03464390e+00  -1.08955611e-01   4.27987106e-02 ...,\n",
      "      2.66289295e-01  -8.85309893e-02   4.17684471e-01]\n",
      "   [ -5.75026525e-01   9.64486380e-01  -2.50498932e-01 ...,\n",
      "     -1.69981671e-01   3.83626229e-02  -4.29510827e-01]\n",
      "   ..., \n",
      "   [ -5.05453794e-01  -8.63296574e-01  -9.17132820e-02 ...,\n",
      "      3.77458619e-01  -1.95137555e-01  -5.17759730e-01]\n",
      "   [ -2.58788723e-01  -9.64247429e-01  -7.14861086e-02 ...,\n",
      "     -1.42263971e-01   1.91776053e-01   5.46139432e-01]\n",
      "   [ -2.64195517e-01   6.57731518e-01   9.02944947e-02 ...,\n",
      "     -3.68829995e-01   3.83041570e-01  -2.01499939e-01]]\n",
      "\n",
      "  [[ -3.65994417e+00  -5.14173995e+00  -4.96577030e+00 ...,\n",
      "     -4.51097770e+00  -4.49018481e+00  -4.99095697e+00]\n",
      "   [ -4.11705978e+00  -3.84009698e+00  -4.76372528e+00 ...,\n",
      "     -5.18592852e+00  -5.22249345e+00  -4.80849165e+00]\n",
      "   [ -3.97032701e+00  -3.38993575e+00  -4.60564690e+00 ...,\n",
      "     -3.40043616e+00  -4.73042337e+00  -5.62832834e+00]\n",
      "   ..., \n",
      "   [ -4.71954725e+00  -2.77478101e+00  -1.26896644e+00 ...,\n",
      "     -3.80103310e+00  -3.90681893e+00  -3.88966040e+00]\n",
      "   [ -4.00748279e+00  -3.81785394e+00  -4.35615060e+00 ...,\n",
      "     -4.77401208e+00  -5.17462886e+00  -4.66860414e+00]\n",
      "   [ -3.03090842e+00  -1.37022436e+00  -2.61072677e+00 ...,\n",
      "     -3.64930527e+00  -3.39860399e+00  -3.79809401e+00]]\n",
      "\n",
      "  ..., \n",
      "  [[ -1.68899562e-01  -4.70391744e-01  -6.97371397e-01 ...,\n",
      "     -5.28510538e-01  -8.69725970e-01  -9.40058672e-01]\n",
      "   [ -1.13698729e-01  -2.39520724e-01  -1.05513776e+00 ...,\n",
      "      1.89672299e-02  -5.85293545e-01  -1.24348531e+00]\n",
      "   [  4.84735384e-01  -2.85855296e-01  -2.31752558e-01 ...,\n",
      "      2.45801093e-01  -1.48229439e+00  -3.27884453e-01]\n",
      "   ..., \n",
      "   [ -1.15616940e+00  -3.67509939e-01  -1.53234489e+00 ...,\n",
      "      3.00053739e-01  -2.14880543e+00  -1.62754632e-01]\n",
      "   [  3.90369861e-01  -1.21054544e+00  -2.43554270e-02 ...,\n",
      "     -1.52735065e+00   1.01189873e+00  -9.24407381e-01]\n",
      "   [ -6.92695001e-01   7.25373323e-01  -4.75248336e-01 ...,\n",
      "      8.00767456e-01   1.86470517e-01  -1.68280526e+00]]\n",
      "\n",
      "  [[  1.00908039e+00   1.18807713e+00   1.30922008e+00 ...,\n",
      "      1.39107018e+00   2.26549537e+00   2.93546801e+00]\n",
      "   [  3.13643280e+00   1.43321484e+00   1.99360973e+00 ...,\n",
      "      1.85552427e+00   9.63954788e-01   1.18764058e+00]\n",
      "   [  1.73749805e+00   2.13676041e+00   2.66441862e+00 ...,\n",
      "      2.95361881e+00   2.92212581e+00   2.39243630e+00]\n",
      "   ..., \n",
      "   [  2.37215457e+00   5.14820833e-01   1.65617652e-01 ...,\n",
      "      1.73166437e+00   1.92869916e+00   2.61471371e+00]\n",
      "   [  2.93004702e+00   1.20636201e+00   1.77732324e+00 ...,\n",
      "      1.48198130e+00   1.97564921e+00   1.10872634e+00]\n",
      "   [  8.97655375e-01   1.19172669e+00   1.02551056e+00 ...,\n",
      "      1.31002310e+00   1.73514415e+00   4.43609674e-01]]\n",
      "\n",
      "  [[  1.56064707e-01   4.11192449e-02  -1.37242171e+00 ...,\n",
      "      2.52051019e-01  -3.18872694e-01  -7.55935752e-01]\n",
      "   [  1.35216298e-01   6.02522093e-01   7.14582454e-01 ...,\n",
      "      2.80997509e-01  -2.69425698e-01   1.67536314e+00]\n",
      "   [  9.27890678e-01   1.72920391e+00   5.26195342e-01 ...,\n",
      "      1.34184756e+00   1.13691001e+00  -9.60607615e-01]\n",
      "   ..., \n",
      "   [  2.85855771e-01  -5.08133719e-01   1.19018943e-01 ...,\n",
      "      1.65176111e-01   7.94100231e-01   5.13618884e-01]\n",
      "   [  9.61006454e-01   1.08911135e+00  -6.81093900e-02 ...,\n",
      "     -8.57343171e-02  -9.72893105e-02  -1.80669722e-01]\n",
      "   [  8.79052882e-01  -4.41791561e-01   1.12758501e+00 ...,\n",
      "      1.81297062e+00  -1.18995210e+00  -6.84469087e-02]]]]\n",
      "max 4.78987174592\n",
      "min -6.67566047666\n",
      "##############################\n",
      "weight shape (20, 1, 5, 5)\n",
      "kermax 0.996485495516\n",
      "kermin -0.998557337864\n",
      "[[[[-1.16666667 -1.96666667 -0.5        ..., -1.56666667 -2.43333333\n",
      "    -1.96666667]\n",
      "   [-1.3        -1.63333333 -1.7        ..., -1.2        -2.13333333\n",
      "    -2.33333333]\n",
      "   [-1.13333333 -1.63333333 -1.23333333 ..., -2.16666667 -2.63333333 -1.2       ]\n",
      "   ..., \n",
      "   [-1.56666667 -0.6        -2.13333333 ..., -1.36666667 -3.56666667\n",
      "    -3.66666667]\n",
      "   [-2.06666667 -1.03333333  0.         ..., -2.         -1.3        -2.03333333]\n",
      "   [-1.23333333 -1.03333333 -1.03333333 ..., -0.06666667 -0.96666667\n",
      "    -1.43333333]]\n",
      "\n",
      "  [[-0.23333333 -0.06666667  0.96666667 ...,  0.86666667  0.63333333\n",
      "     0.16666667]\n",
      "   [-1.03333333 -0.1         0.03333333 ...,  0.23333333 -0.03333333  0.4       ]\n",
      "   [-0.53333333  0.93333333 -0.23333333 ..., -0.13333333  0.03333333\n",
      "    -0.43333333]\n",
      "   ..., \n",
      "   [-0.5        -0.83333333 -0.06666667 ...,  0.33333333 -0.16666667 -0.5       ]\n",
      "   [-0.23333333 -0.9        -0.06666667 ..., -0.13333333  0.16666667\n",
      "     0.53333333]\n",
      "   [-0.26666667  0.6         0.06666667 ..., -0.33333333  0.36666667 -0.2       ]]\n",
      "\n",
      "  [[-3.46666667 -4.93333333 -4.73333333 ..., -4.33333333 -4.26666667\n",
      "    -4.76666667]\n",
      "   [-3.96666667 -3.66666667 -4.56666667 ..., -5.         -5.03333333\n",
      "    -4.63333333]\n",
      "   [-3.73333333 -3.13333333 -4.36666667 ..., -3.16666667 -4.5        -5.43333333]\n",
      "   ..., \n",
      "   [-4.53333333 -2.6        -1.13333333 ..., -3.6        -3.7        -3.66666667]\n",
      "   [-3.83333333 -3.63333333 -4.13333333 ..., -4.56666667 -4.96666667\n",
      "    -4.46666667]\n",
      "   [-2.8        -1.23333333 -2.5        ..., -3.46666667 -3.2        -3.6       ]]\n",
      "\n",
      "  ..., \n",
      "  [[-0.13333333 -0.46666667 -0.66666667 ..., -0.53333333 -0.83333333 -0.9       ]\n",
      "   [-0.06666667 -0.2        -1.         ...,  0.03333333 -0.56666667\n",
      "    -1.23333333]\n",
      "   [ 0.46666667 -0.26666667 -0.2        ...,  0.2        -1.43333333 -0.3       ]\n",
      "   ..., \n",
      "   [-1.2        -0.33333333 -1.46666667 ...,  0.3        -2.06666667 -0.1       ]\n",
      "   [ 0.36666667 -1.16666667 -0.03333333 ..., -1.56666667  1.03333333 -0.9       ]\n",
      "   [-0.66666667  0.7        -0.36666667 ...,  0.76666667  0.2        -1.66666667]]\n",
      "\n",
      "  [[ 0.9         1.1         1.2        ...,  1.33333333  2.16666667\n",
      "     2.83333333]\n",
      "   [ 3.03333333  1.33333333  1.9        ...,  1.73333333  0.9         1.13333333]\n",
      "   [ 1.63333333  2.06666667  2.6        ...,  2.86666667  2.83333333\n",
      "     2.33333333]\n",
      "   ..., \n",
      "   [ 2.3         0.5         0.1        ...,  1.6         1.86666667\n",
      "     2.53333333]\n",
      "   [ 2.83333333  1.13333333  1.7        ...,  1.36666667  1.9         1.03333333]\n",
      "   [ 0.8         1.13333333  0.96666667 ...,  1.2         1.66666667\n",
      "     0.36666667]]\n",
      "\n",
      "  [[ 0.16666667  0.03333333 -1.4        ...,  0.23333333 -0.33333333\n",
      "    -0.73333333]\n",
      "   [ 0.1         0.6         0.7        ...,  0.26666667 -0.23333333\n",
      "     1.66666667]\n",
      "   [ 0.96666667  1.7         0.53333333 ...,  1.33333333  1.1        -0.93333333]\n",
      "   ..., \n",
      "   [ 0.26666667 -0.5         0.1        ...,  0.1         0.83333333  0.5       ]\n",
      "   [ 0.86666667  1.06666667  0.         ..., -0.1        -0.1        -0.13333333]\n",
      "   [ 0.86666667 -0.43333333  1.06666667 ...,  1.8        -1.13333333\n",
      "    -0.06666667]]]]\n",
      "max 4.73333333333\n",
      "min -6.53333333333\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "False"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "## Testbench for a single conv layer\n",
    "net1 = {}\n",
    "net1['input_1'] = InputLayer((None, 1, 28, 28))\n",
    "net1['conv_1'] = ConvLayer(net1['input_1'], num_filters=20, filter_size=5, pad=0, nonlinearity=linear)\n",
    "weight = np.random.rand(20, 1, 5, 5)*2-1\n",
    "#weight = weight[...,::-1,::-1]\n",
    "print('weight shape',weight.shape)\n",
    "\n",
    "net1['conv_1'].W.set_value(weight)\n",
    "input = np.random.rand(1, 1, 28, 28)\n",
    "print('input shape',input.shape)\n",
    "conv_output_test = lasagne.layers.get_output(net1['conv_1'], floatX(input), deterministic=True).eval()\n",
    "print(conv_output_test)\n",
    "print('max', np.amax(conv_output_test))\n",
    "print('min', np.amin(conv_output_test))\n",
    "print('##############################')\n",
    "FPGALoadW(weight, 1, 28, 24, 0)\n",
    "FPGA_output = FPGAQuickTest(input, 1, 24, 20)\n",
    "print(FPGA_output)\n",
    "print('max', np.amax(FPGA_output))\n",
    "print('min', np.amin(FPGA_output))\n",
    "np.allclose(conv_output_test, FPGA_output)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[  4.87205496   0.           0.           0.          10.32939182   0.\n",
      "    3.73065467   0.           3.32166961   8.04596126   2.47765243\n",
      "    2.38071792   0.           0.          10.23824185  11.37844921   0.\n",
      "    2.49904565   6.67317322   7.72101975   0.          16.22842281\n",
      "    4.21925851   1.71412087  13.86398149   0.           6.47131831   0.\n",
      "    0.           0.          11.01161799  16.50425193  10.09142138   0.\n",
      "    8.71200305   0.          17.73560793   7.26002949   0.           0.\n",
      "    0.           0.           0.           3.64882534   8.84798804\n",
      "    0.51790838   0.          12.24283568   0.          10.07998283   0.\n",
      "   13.93915632   0.           1.31342381   2.37565862   0.18018827\n",
      "   15.77413368   0.           0.           3.52941475   4.3752688\n",
      "    0.96245774   5.99787461   0.           0.           0.           7.57239814\n",
      "   11.34722104   0.           7.44900567   8.85447856   0.           3.63706984\n",
      "    4.4351584    6.1741987    5.12098359   0.           0.           0.54486455\n",
      "    0.           2.34716758  11.65961668  14.37530636   2.73037676   0.\n",
      "   11.02818577   0.           1.05222788   0.           0.           0.\n",
      "   11.79985969   0.           2.18722952   8.07016441   0.           0.48128377\n",
      "    0.42685334   0.           0.           0.           7.97839663   0.\n",
      "    0.           0.58764638   8.87131068   0.          15.18896369   0.\n",
      "    0.           5.69908405   0.           0.           0.           0.\n",
      "    0.           8.1806746   10.6865666    5.4502488    0.          15.43516532\n",
      "   11.31803067   0.          15.8883904    0.           0.           0.\n",
      "    0.           0.           0.           0.           3.36289894\n",
      "   10.05100219   0.           0.           0.           0.           1.36648831\n",
      "    0.           2.34897277   0.           0.           0.16635098\n",
      "    2.03511467   0.           0.           0.82106747   0.          13.56543568\n",
      "    5.50418203   0.           6.48636786  14.66925314   0.           0.\n",
      "    9.72713839   4.6732693    0.66350285   6.13598948   0.           0.\n",
      "    0.           0.           7.71326117  21.66236202   6.07559027\n",
      "   11.30994027   0.03813952   0.           4.55986687   0.36733798\n",
      "   18.66398918   0.           5.73153027   0.           6.62628006   0.\n",
      "    0.           4.79217688   3.05292392   5.97852612   2.73634945   0.\n",
      "    0.           0.          11.81853997   0.           6.1895648    0.\n",
      "    0.           0.           0.          10.3059492    5.61476791\n",
      "    4.7638238    8.01105809   1.1456242    5.62021886   0.           0.38868603\n",
      "    1.33126799   0.42378301   0.           0.          12.38176997\n",
      "   10.35476552   0.           0.           0.          14.59775052   0.\n",
      "    0.           0.           5.85772319   0.           0.           0.\n",
      "    0.           0.78042736   0.           2.98850167   4.02550251\n",
      "    8.51495478   0.           0.           0.           0.           0.\n",
      "   13.9854325   11.85832479   0.           0.           9.23101527   0.\n",
      "   11.75796767   5.35753142   1.41690737  11.95718263  17.79470972\n",
      "    9.28500759   0.           0.           0.21147603   2.15857456   0.\n",
      "    0.          19.0844405   22.15722801   0.           4.17137518   0.\n",
      "    6.92154393   1.69509028  18.6839166    5.69489588   0.           0.\n",
      "    0.          11.88879818   5.15587561   0.          12.85335112\n",
      "    3.70553621   0.           0.           5.64413022   6.60557047\n",
      "    3.96497751   0.           4.60287163   0.           0.58812798   0.\n",
      "   11.1157155    0.           0.           0.           9.8155275    0.\n",
      "    0.          16.20489183   0.           0.           4.14233276   0.\n",
      "    4.34341638   8.46896904   0.           8.37185049  10.31071701   0.\n",
      "    0.10688457   0.           5.97306068   0.           4.89753857   0.\n",
      "    9.26196222   0.           9.58950062  13.38471899   2.16669255   0.\n",
      "    5.34838811   6.96073294   0.           0.           0.          15.08913938\n",
      "    2.51803672   7.88308158  12.21421887  10.20202114  10.73308962\n",
      "    3.52093679   0.           4.47425126   7.85916816   0.           0.\n",
      "    6.90158195   8.29529849   0.           0.           0.           0.\n",
      "   17.66852546   0.           0.           2.93399414   0.           0.\n",
      "    5.09620742   0.           0.           7.2849421    0.           0.\n",
      "    0.          10.18709824   2.5744383    0.           4.08406648   0.\n",
      "    6.83491966   9.35418875   0.           0.           0.           0.\n",
      "    4.95455747   0.           5.09100722   0.           8.41360688   0.\n",
      "    8.46274502   0.           6.89214509   6.67986715   0.           0.\n",
      "    0.           0.           2.28805299   0.           8.28019421\n",
      "   12.75343376   0.           0.           2.51653061   0.           0.82721538\n",
      "    0.           8.91417208   0.           5.68516714   3.11026239\n",
      "    7.29585777   0.           2.14074919   9.47591846   0.           0.\n",
      "   14.82286938   0.           0.           0.           0.           7.88781567\n",
      "   10.51508377   0.           0.           0.45894944   3.59168604   0.\n",
      "   14.03073171   6.76643939   0.           3.1323703   12.24137663\n",
      "    2.94689514   0.           0.           0.           7.25376755\n",
      "    3.42208125  19.07386738   0.           9.51545039   0.           0.\n",
      "   13.94424881  10.40733373   0.           0.           0.           0.\n",
      "    3.17219076   0.           5.35096481  10.96430379  11.77482422\n",
      "   11.30419229   0.           0.           0.          19.08975336\n",
      "    6.08621084   4.37618314   0.           1.98863682   7.90072457\n",
      "   11.825676     0.           0.28749063   0.           0.           0.\n",
      "    0.           0.           0.           6.91871997  15.02426763\n",
      "    3.41454367   6.40912401  11.50653594   0.           0.           0.31042584\n",
      "    0.           0.           3.18485489   0.           0.           0.\n",
      "    0.           0.           1.73935809   0.           4.51500803\n",
      "    9.51558738   4.15108236   4.26826456   0.           1.67253442\n",
      "    5.32168599   0.           4.73830834   0.           1.47740501   0.\n",
      "    0.           0.           0.           0.           0.          12.76195807\n",
      "    0.           0.           0.03329534   0.           0.48526704   0.\n",
      "    0.           3.19732557  20.76907562   0.           0.           1.19507225\n",
      "    0.           0.           3.39872442   0.           3.65393208\n",
      "   14.42782519  12.05240166   0.           0.           9.11073052]]\n",
      "max 22.157228007\n",
      "min 0.0\n",
      "##############################\n",
      "weight shape (500, 50, 4, 4)\n",
      "kermax 0.999999553699\n",
      "kermin -0.999996939193\n",
      "[[  4.8203125   0.          0.          0.         10.1328125   0.\n",
      "    3.7265625   0.          3.296875    8.0234375   2.421875    2.4765625\n",
      "    0.          0.         10.046875   11.3359375   0.          2.4765625\n",
      "    6.5234375   7.5625      0.         16.          4.140625    1.6484375\n",
      "   13.6953125   0.          6.4765625   0.          0.          0.\n",
      "   10.7734375  16.1171875  10.0703125   0.          8.59375     0.         17.5\n",
      "    7.28125     0.          0.          0.          0.          0.          3.6875\n",
      "    8.8828125   0.4140625   0.         12.2109375   0.          9.9296875\n",
      "    0.         13.6640625   0.          1.4921875   2.28125     0.1875\n",
      "   15.6484375   0.          0.          3.53125     4.234375    1.0234375\n",
      "    5.921875    0.          0.          0.          7.5625     11.2578125\n",
      "    0.          7.5078125   8.671875    0.          3.6328125   4.3359375\n",
      "    5.953125    5.0625      0.          0.          0.484375    0.\n",
      "    2.328125   11.4921875  14.296875    2.671875    0.         10.890625\n",
      "    0.          1.03125     0.          0.          0.         11.578125\n",
      "    0.          2.1015625   8.09375     0.          0.40625     0.4609375\n",
      "    0.          0.          0.          8.0078125   0.          0.\n",
      "    0.5390625   8.7421875   0.         15.03125     0.          0.\n",
      "    5.5703125   0.          0.          0.          0.          0.\n",
      "    8.03125    10.6875      5.3359375   0.         15.203125   11.2421875\n",
      "    0.         15.65625     0.          0.          0.          0.          0.\n",
      "    0.          0.          3.4453125   9.9765625   0.          0.          0.\n",
      "    0.          1.25        0.          2.4296875   0.          0.\n",
      "    0.2109375   1.9921875   0.          0.          0.7265625   0.\n",
      "   13.4453125   5.390625    0.          6.40625    14.4609375   0.          0.\n",
      "    9.546875    4.609375    0.703125    6.1484375   0.          0.          0.\n",
      "    0.          7.6640625  21.4765625   5.9296875  11.171875    0.078125\n",
      "    0.          4.453125    0.2734375  18.5         0.          5.6171875\n",
      "    0.          6.453125    0.          0.          4.7578125   2.9921875\n",
      "    5.890625    2.59375     0.          0.          0.         11.671875\n",
      "    0.          6.25        0.          0.          0.          0.\n",
      "   10.1171875   5.453125    4.75        7.8046875   1.2578125   5.6015625\n",
      "    0.          0.453125    1.234375    0.375       0.          0.\n",
      "   12.2890625  10.2109375   0.          0.          0.         14.4453125\n",
      "    0.          0.          0.          5.84375     0.          0.          0.\n",
      "    0.          0.7578125   0.          2.9609375   3.984375    8.3515625\n",
      "    0.          0.          0.          0.          0.         13.7890625\n",
      "   11.828125    0.          0.          9.21875     0.         11.6328125\n",
      "    5.3125      1.4140625  11.7890625  17.625       9.203125    0.          0.\n",
      "    0.1171875   2.1953125   0.          0.         18.9140625  21.875       0.\n",
      "    4.09375     0.          6.84375     1.6640625  18.4453125   5.7421875\n",
      "    0.          0.          0.         11.6953125   5.0703125   0.\n",
      "   12.5703125   3.6953125   0.          0.          5.6328125   6.53125\n",
      "    3.9453125   0.          4.59375     0.          0.5625      0.\n",
      "   10.9453125   0.          0.          0.          9.546875    0.          0.\n",
      "   16.0859375   0.          0.          3.9921875   0.          4.2578125\n",
      "    8.3984375   0.          8.2890625  10.203125    0.          0.1484375\n",
      "    0.          5.8828125   0.          4.7734375   0.          9.171875\n",
      "    0.          9.546875   13.25        2.0703125   0.          5.1640625\n",
      "    6.7578125   0.          0.          0.         14.984375    2.5234375\n",
      "    7.71875    12.1328125  10.15625    10.640625    3.5         0.\n",
      "    4.4296875   7.640625    0.          0.          6.8046875   8.1953125\n",
      "    0.          0.          0.          0.         17.3671875   0.          0.\n",
      "    2.953125    0.          0.          5.0859375   0.          0.          7.3125\n",
      "    0.          0.          0.         10.0703125   2.5078125   0.\n",
      "    4.015625    0.          6.765625    9.2109375   0.          0.          0.\n",
      "    0.          4.859375    0.          4.9765625   0.          8.265625\n",
      "    0.          8.4453125   0.          6.828125    6.6328125   0.          0.\n",
      "    0.          0.          2.21875     0.          8.296875   12.65625\n",
      "    0.          0.          2.46875     0.          0.8203125   0.\n",
      "    8.9140625   0.          5.65625     3.203125    7.2265625   0.\n",
      "    2.171875    9.3203125   0.          0.         14.578125    0.          0.\n",
      "    0.          0.          7.8359375  10.375       0.          0.\n",
      "    0.4296875   3.6953125   0.         13.8984375   6.703125    0.\n",
      "    3.0546875  12.1171875   3.0078125   0.          0.          0.\n",
      "    7.03125     3.421875   18.9140625   0.          9.453125    0.          0.\n",
      "   13.8359375  10.3203125   0.          0.          0.          0.\n",
      "    3.1640625   0.          5.3671875  10.96875    11.6328125  11.234375\n",
      "    0.          0.          0.         18.8203125   6.0390625   4.375       0.\n",
      "    1.8359375   7.8515625  11.7109375   0.          0.265625    0.          0.\n",
      "    0.          0.          0.          0.          6.875      14.78125\n",
      "    3.4921875   6.3984375  11.3203125   0.          0.          0.3125      0.\n",
      "    0.          3.140625    0.          0.          0.          0.          0.\n",
      "    1.7421875   0.          4.40625     9.4140625   4.0859375   4.15625\n",
      "    0.          1.6328125   5.3359375   0.          4.5703125   0.\n",
      "    1.34375     0.          0.          0.          0.          0.          0.\n",
      "   12.6640625   0.          0.          0.0859375   0.          0.4296875\n",
      "    0.          0.          3.0546875  20.5078125   0.          0.\n",
      "    1.2421875   0.          0.          3.2890625   0.          3.59375\n",
      "   14.2890625  11.9140625   0.          0.          9.1015625]]\n",
      "max 21.875\n",
      "min 0.0\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "False"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "## Testbench for a single ip layer\n",
    "net2 = {}\n",
    "net2['input_2'] = InputLayer((None, 50, 4, 4))\n",
    "net2['ip_2'] = DenseLayer(net2['input_2'], num_units=500, nonlinearity=rectify)\n",
    "weight = np.random.rand(800,500)*2-1\n",
    "#weight = np.arange(32)/32\n",
    "#weight = weight.reshape(8,4)\n",
    "#print('weight (caffe)',weight)\n",
    "\n",
    "net2['ip_2'].W.set_value(weight)\n",
    "input = np.random.rand(1, 50, 4, 4)\n",
    "#input = np.arange(8)/8\n",
    "#input = input.reshape(1,2,2,2)\n",
    "#print('input (caffe)',input)\n",
    "conv_output_test = lasagne.layers.get_output(net2['ip_2'], floatX(input), deterministic=True).eval()\n",
    "print(conv_output_test)\n",
    "print('max', np.amax(conv_output_test))\n",
    "print('min', np.amin(conv_output_test))\n",
    "print('##############################')\n",
    "weight = np.transpose(weight)\n",
    "weight = weight.reshape(500, 50, 4, 4)\n",
    "#print('weight (fpga)',weight)\n",
    "#weight = weight[...,::-1,::-1]\n",
    "FPGALoadW(weight, 3, 4, 1, 0, flip_filters=False)\n",
    "#print('input (fpga)',input)\n",
    "FPGA_output = FPGAQuickTest(input, 1, 1, 500)\n",
    "print(FPGA_output.reshape(1,500))\n",
    "print('max', np.amax(FPGA_output))\n",
    "print('min', np.amin(FPGA_output))\n",
    "np.allclose(conv_output_test, FPGA_output)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Make predictions on the test data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Elapsed Test Time:  5.661279272999998\n"
     ]
    }
   ],
   "source": [
    "start_time = time.process_time()\n",
    "prob = np.array(lasagne.layers.get_output(net['prob'], floatX(test_data[0:20]), deterministic=True).eval())\n",
    "predicted = np.argmax(prob, 1)\n",
    "end_time = time.process_time()\n",
    "print(\"Elapsed Test Time: \", end_time-start_time)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Check our accuracy\n",
    "We expect around 90%"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[7 2 1 0 4 1 4 9 5 9 0 6 9 0 1 5 9 7 3 4]\n",
      "[7 2 1 0 4 1 4 9 5 9 0 6 9 0 1 5 9 7 3 4]\n",
      "1.0\n"
     ]
    }
   ],
   "source": [
    "accuracy = np.mean(predicted == test_label[0][0:20])\n",
    "print(predicted)\n",
    "print(test_label[0][0:20])\n",
    "print(accuracy)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Double check\n",
    "Let's compare predictions against Caffe"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "net_caffe.blobs['data'].reshape(1000, 3, 32, 32)\n",
    "net_caffe.blobs['data'].data[:] = data['whitened']\n",
    "prob_caffe = net_caffe.forward()['pool3'][:,:,0,0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.allclose(prob, prob_caffe)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Graph some images and predictions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA5sAAABuCAYAAACp3AcnAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xd8VFX6P/DPEBAikaahFylGAgkkIEU2soo0qUZhDSAq\nTXYVRWzo6rrYsKBo6GUtgEhRUZDqokQhwvIDgy4BzA8QQpCqSAtSzHz/YM/JM8yEJDN3zs3c+bz3\nxYvHM5m5z55c7syd+9znuNxuN4iIiIiIiIisVMruBIiIiIiIiMh5gnKyGRkZedDlcrn9+RMZGXkw\nGDk5lb9zzXkuPs61OZxrM3isNodzbQ6PH+Zwrs3hXJvBY7X1XMEoo3W5XG5/X9flcsHtdrssTsmx\n/J1rznPxca7N4VybwWO1OZxrc3j8MIdzbQ7n2gweq63HMloiIiIiIiKyHE82iYiIiIiIyHI82SQi\nIiIiIiLLGTvZzMrKQmJiIlq0aIHExERUrFgREyZMMLX5sJKTk4MOHTqgadOmiI+P5zwH0ZAhQ1Ct\nWjU0a9bM7lQcb+XKlWjcuDFiYmLw2muv2Z2O4+Xl5aFFixbo1auX3ak4Fo8fZnGfNiM1NRXx8fH8\n/GHAtddei+bNmyMxMRGtW7e2Ox1H437tP2MnmzExMcjIyMB3332HzZs3o3z58khOTja1+bBSunRp\njB8/HpmZmVi/fj0mT56MHTt22J2WIw0aNAirVq2yOw3Hy8vLw4gRI7Bq1SpkZmZi3rx53KeDLDU1\nFU2aNLE7DUfj8cMs7tPBl5mZiXfeeQebNm3Cli1bsHTpUuzevdvutByrVKlSSEtLQ0ZGBjZu3Gh3\nOo7F/TowtpTRrl69Gg0bNkSdOnXs2LzjVa9eHQkJCQCAqKgoxMbGYv/+/TZn5UxJSUmoXLmy3Wk4\n3saNG3HdddehXr16KFOmDFJSUrB48WK703KsnJwcLF++HEOHDrU7FUfj8cMc7tNmbN++HW3atEHZ\nsmURERGB9u3bY9GiRXan5Vhutxt5eXl2p+F43K8DY8vJ5oIFC9CvXz87Nh129uzZgy1btqBNmzZ2\np0Lkt/3793t8OVW7dm1+gRJEo0aNwrhx4+BysYM7OQP3aTPi4uKwdu1aHDt2DLm5uVi+fDn27dtn\nd1qO5XK50KlTJ7Rq1QozZ860Ox3H4n4dmNKmN3j+/HksWbIEr776qulNh51Tp06hT58+SE1NRVRU\nlN3pEFEIWLZsGapVq4aEhASkpaUhGGsxE5nEfdqcxo0bY/To0ejUqROioqKQmJiIiIgIu9NyrPT0\ndNSoUQNHjhxBp06dEBsbi6SkJLvTchzu14ExfmVzxYoVaNmyJaKjo01vOqxcuHABffr0wcCBA9G7\nd2+70yEKSK1atZCdna3/OycnB7Vq1bIxI+dKT0/HkiVL0KBBA/Tr1w9r1qzBPffcY3daRH7jPm3W\noEGDsGnTJqSlpaFSpUqIiYmxOyXHqlGjBgAgOjoaycnJvG8ziLhfB8Dtdlv+5+LL+paSkuJ+//33\nC3wcQNj/sWKuBw4c6B41ahTn2aJ5vtxc//TTT+64uDjOdRDn+sKFC+6GDRu69+zZ4z579qy7efPm\n7m3btnGuA5zrS+f5Umlpae6ePXtyvw7iXBd2/OBcW7tfc5+2bp4LmuvDhw+73W63e+/eve7Y2Fj3\n8ePHOddBmOvTp0+7T5486Xa73e5Tp06527Vr5161ahXnOsC5vnSeuV8HPtdGr2zm5uZi9erVuOOO\nO0xuNuykp6dj7ty5+Oqrr/RyMytXrrQ7LUfq378/2rVrh6ysLNStWxfvvfee3Sk5UkREBCZNmoTO\nnTujadOmSElJQWxsrN1pEQWExw9yojvvvBNxcXHo3bs3pkyZggoVKtidkiMdOnQISUlJSExMRNu2\nbdGzZ0907tzZ7rQci/u1/1z/O4u39kVdLre/r8ub9wG3213kSfB3rjnPxZtngHMdCM61OSaOH/97\nrl/PcxLOtTl8XzSDx2pzONfm8Fhtjq+5tqUbLRERERERETkbTzaJiIiIiIjIcjzZJCIiIiIiIsvx\nZJOIiIiIiIgsVzoYL1quXLlDLpermj/PLVu2LM6ePWt1So7l71wHOs+PP/64jiMjI3XcrFkzAECf\nPn18Pm/q1KkAgPXr1+uxOXPm+J2HSXbNdTjiXJvBY7U5nGtzePwwh3NtDufaDB6rrReUk80zZ85U\nV7HL5SpWSyf+korH37nmPBcf59ocNdc8fgQXj9XmcK7N4bHaHM61OXxfNIPHauuxjJaIiIiIiIgs\nF5R1Nj02UMxvBaj4ay8pwZ7rBQsW6LigMtmi2rVrl447duyo4+zs7IBetzj8nWcgtPbrmJgYHe/Y\nsUPHI0eO1PHEiRODmkNJ3aeLonz58joeN24cAGD48OF6bPPmzTru27evjvfu3WsgO2+hPNehhnNt\nDufajHB5XywJwnGfrly5MgCgbt26hf6seg8dNWqUHtu6dauOs7KydPz9999f9rXCca7twnU2iYiI\niIiIyAiebBIREREREZHlgtIgiJxFlc8WpXRWlWmuWrVKjzVo0EDHPXv2BAA0bNhQjw0YMEDHr7zy\nSmDJkpfExEQd5+Xl6TgnJ8eOdEJOjRo1dDxs2DAAnvPYsmVLHffo0UPHkydPNpBdaGrRooWOFy1a\npONrr73Wsm107txZx9u3b9fxvn37LNtGuFDHbQBYsmQJAGDEiBF6bNq0aTr+448/zCVWAlStWlXH\nCxcu1PG3336r4xkzZgAA9uzZE5QcKlasqOP27dsDAFauXKnHzp8/H5TtEhWke/fuOu7Vq5eOb775\nZgBAo0aNCn0NVSZbr149PVa2bFmfPxsREeFPmmQIr2wSERERERGR5XiySURERERERJZjGS35dMMN\nN+g4OTnZ6/HMzEwdyxKJo0ePAgBOnTqlx6644godb9iwAQDQvHlzPXb11VdbkDEVJCEhQcenT5/W\n8aeffmpHOiEhOjpax7NmzbIxE2fq0qWLjgsqiwqULP0cPHiwjlNSUoKyPaeRx+UpU6Z4PT5p0iQd\nv/vuuzo+c+ZMcBMrIVRXTfleKMtZDx06pONglM/KbcmO2OrYJcv7d+7cafn2S5IKFSroWN2KExcX\np8dkx3uWFFtD3gr14IMPAsi/zQQAIiMjdexy+dfgWHbSp9DGK5tERERERERkuRJ/ZVM2pZHfmvz8\n8886/v333wEAc+fO1WMHDx7UsdO/1QsG2RRFfSslv8GVVyYOHDhw2dd67LHHdNykSROvx5ctW+Z3\nnlQw9c2ubOQxZ84cu9Ip8R5++GEd33777Tpu3bp1kV9DNecAgFKlLn6XJ9f/+uabbwJJMeSVLn3x\nLadbt25B35a82vPoo4/qWK2bKq/ykze5L9euXdvr8Xnz5ulYvQc73TXXXKNj1TivSpUqekxeAX7o\noYeCmsuzzz6r4/r16+tYrQHs9M89srHgyy+/rOM6dep4/ay88vnLL78EN7EwIY8Jcs3uQMm1wOVn\nTrpINVaSxyJZfagaMAH5jQxlA7f09HQdmzxG8MomERERERERWY4nm0RERERERGS5El9G+/rrr+u4\nsDXYVPkIAJw8eVLHwboUL9cpVHlu2rQpKNsy7fPPP9exumwv5/TXX38t8mvJhhxlypSxIDsqisaN\nGwPILxsE8ku/yNtbb72lY7mOZnHccccdXvHevXv12F133aVjWeYZLm655RYAwI033qjH5DHeSqqB\nC+BZvn/llVcCYBmtL7JZ0zPPPHPZn5Ul+W63O2g5lSRyfVhZrqa88MILQc+hadOmADxvT5HN3px8\njJelm2+//baOZTMrX/vixIkTdaxuKynOZ5hwIUszVWmsLLuUa7eePXtWx8ePHwfgeUyVnzu++OIL\nHW/duhUA8J///EePZWRk6Fg2GAvnY7RscCVvhVKfK+TvqjBt2rTR8YULF3T8448/6njdunUAPEui\nz507V4yML49XNomIiIiIiMhyPNkkIiIiIiIiy5X4MlrZgbZZs2Y63r59u45jY2MBFFzi0rZtWx3v\n27cPgO+OZZdSl5uPHDmix2SXVik7OxuAc8poJVkGWFRPPPGEjn2tlSRLKGRM1nnyyScBeP7+nLh/\nBmr58uUA8rvHFpfsbijXl61Xrx4Az06RGzdu1HFERIRf2ws1shxIdTDdtWuXHhs7dmxQttu7d++g\nvK6TxcfH61iu0yip98UVK1YYycluVatW1fGdd97p9fiQIUN0LD8rWEmVzgLA6tWrvR6XZbTydhen\nefzxx3UsuwAXRt6+0LVrVwCeHWxlma2VpYOhoKByV7UWuq911oH8NdOB/M/ecj3ZunXr6ljecubv\nLSpOps5t1HqlgOc+K7spK/v379fx2rVrdfzTTz/pWH0GlLfsyO768t+Q6hAvu+fLLraB4pVNIiIi\nIiIishxPNomIiIiIiMhyJb6M9ssvv/QZS7JDliI7ESYkJOhYXU5u1apVodtWC1VnZWXpMVm+Ky9B\ny7KwcNajRw8Anl35rrjiCh0fPnwYAPD000/rsdzcXEPZOZ/s2HzDDTcA8Nx/w7m7m/TnP/9Zx9df\nfz0Az/Kewkp9ZHmJLD1SXfkAoEOHDgAK7ur5t7/9TcdTp04tStohSS4+r0q2VCkb4Fl6HCh5TJa/\nY5ZuFY2vMtFLyf09HLz55ps6vvvuu3WsPkt89NFHQc/hpptu0nG1atUAAO+//74e++CDD4Keg53U\nLQmDBg3y+fgPP/yg40OHDgEAOnbs6PNnK1asCMCzJHfu3Lk6PnjwYGDJhgD5mezDDz/UsSqdBfJv\nb/BVtn0pWT6rqFvLyLfp06frWJUqF9RhVp77/Pe//wUA/P3vf9dj6lzlUu3atQPg+Vnj3Xff1bE8\nN1L/biZPnqzHPvnkEx0HeosAr2wSERERERGR5Ur8lU1/HTt2TMdr1qzxerygq6S+yG975RVT9Q0D\n4Oy1rYpDXU2T35xJap6+/vprYzmFE3k1RwlW04pQI6/6zp8/X8eFrVclGyypb/qef/55PVbQlXn1\nvPvvv1+PRUdH61iuL1muXDkAwKRJk/TY+fPnL5tXSdanTx8dq8YDALBz504AwWtUJa8iy6uZaWlp\nOv7tt9+Csm0naN++vc9x2TSlsPU3nUau2yj3qZ9//hmAtQ1lIiMjdSyvXDzwwANe+QwePNiy7ZZ0\n6grMVVddpcdkUxT5vqeOpf369dNjci4bNmwIAKhevboeW7x4sY5vu+02HTttLc6oqCgAnpVlqhoN\nAI4eParjN954AwArzwKl9kcgv2EPAAwdOlTHLpcLgOdnNVntNG7cOB0XpzpNrT8rmxGOGTNGx7Iq\nVFUPBAuvbBIREREREZHleLJJRERERERElnNsGa0V1PpaU6ZM0WNyLT7ZBMdp5RbF8dlnn+m4c+fO\nXo/Pnj1bx7JZCFlPrpOnyHLNcFa6dP7hrrDSWVnmnZKSomNZZlQYVUb7yiuv6LHx48fr+Morr9Sx\n+h0tWbJEj4Vy07G+ffvqWP7/lMdSK6kS6QEDBuixP/74Q8cvvfSSjkO5PDlYVCMJ9felZOnWli1b\njORU0nXv3h2AZ8MkWaJdnKZfqgy0oPXBpY8//rg4aTpC2bJlAXiWNL/11ls+f1Y1S3nvvff0mDwe\nNWjQwOs5slTUyets3n777QCAp556So/JRj6yEZVsdkf+k/+m5frzqnQWyF8zU96yJ9fkLowsk61T\np46O1WdvtZY44HkroKTymTNnjh6z8pYTXtkkIiIiIiIiy/Fkk4iIiIiIiCzHMtrLePDBBwF4dpCU\nXW5//PFH4zmVFDVq1NCxLL1S5S6y3FCWsFm5ph5dJMut5DpkGRkZAIB///vfxnMKRbJDquz0WJzS\nWV9kaaws8yzKWr+hRK1fBxRcAhis9URVx19ZHi3XRPbVkZzyFbYvOnkd2MKkpqbq+JZbbtFxzZo1\nAXh28JWlcb169SryNtTzZJmotHv3bh3LzqrhQnaWVVQZM+B5K48vqkt+QTZs2KBjJ39G8VUmrz4n\nAEBOTo7JdMKCLHGVt3ZIFy5cAAC0adNGj8mO7o0bN/Z6zpkzZ3QcGxvrM1afXdTavJej1tkM1i0n\nvLJJREREREREluPJJhEREREREVmOZbSX+NOf/qRj2bFLUd28AGDr1q1GciqJ1OL2QP7CsdIHH3yg\n41DuqhkKOnbsqOMqVaroWC3Yq7rzUT7ZVVqRJSxWkqV1cru+cpALLg8cODAo+QSLKqEHgFq1aul4\n3rx5Qd+2WqhdCufjc3H5KjP0t7Oq02zevFnHzZo103FCQgIAoGvXrnpMdpuUC7TPmjXrsttQHSC/\n//57n49/++23Og7H91N1DJGlybL0W5YZqo7sycnJekx24FT7tRwbNmyYjmU3zm3btgWce0kiSzMV\nuf/+85//1PHixYsBsPt0oL766isdy9s55Oe2unXrAgAmTJigxwoqqVeluLI8tyC+ymfz8vJ0/Omn\nn+r44YcfBgAcOHCg0Nf1B69sEhERERERkeVcBZ09W7YBlyu4G7DYyy+/rOOnn34aAPDll1/qsW7d\nuuk4WOu1ud1uV+E/5S3Ycy2/VVy4cKGOy5Qpo+O0tDQAQO/evfVYSb3h3t95BkrWfv3RRx/pWK7T\npGL57ZVdSsI+/cYbb+h45MiRXo/L/dhKDz30kI7lOpvyyqb6tlF+Q+/vFQy75joyMlLHa9eu1bGc\nV9VgxYp1idU6yIDvb2PVN7UAMHny5IC350tJ2K/9lZSUpGO1rqzcJ9U6sUD+OqZ2CuW5Loxa+3Hn\nzp16TF5R6tKli47lFdNgKInvi6piR86PbEgmq0d8faZdvXq1jlXjx6VLl+qx6667TsczZ87U8V//\n+tdA0i6U6X1azY28ulUQ9TPTpk3TY7KRkroaB+T/XjIzM32+VtOmTXW8fv16AOabEZW040elSpV0\nrKooZWXlL7/8omO5FqqqIGrevLkea926dZG3K3+fstmYlWtq+pprXtkkIiIiIiIiy/Fkk4iIiIiI\niCzHBkHwLP+SN0ufO3cOgOdN08EqnS3JVAMgecm9oJJDVfpTUktnnaR69eoAgJtuukmPybVfS0L5\nbEnSs2fPoG9DrsnbpEkTAEVbF0+VxoXy8UWu+yVLgGVp97JlywB4lhMXJi4uTseq3BDwLO30VTpX\nlFKxcCYbu/lqVsX1ec157rnnAHjux6NHj9ZxsEtnSzpVdv+Xv/xFj3388cc6liW1ysSJE3Us51I1\nzFu0aJEek80gZcmyajzmlKZM6laSRx99tNCfVceEBx54QI/J2F9qX1a3XAFASkpKwK8bamTZqq9m\npIWZPXu2jgsqoz158iQAz9/3+++/r+OC1v0MBl7ZJCIiIiIiIsvxZJOIiIiIiIgsxzJaeK6NlZiY\nqGO1TqFc4yocPfbYYwA817WSPvvsMx3LkmMKrvvuuw+AZ1fOFStW2JQNAcAzzzyjY9X1sCB79uzR\n8b333gvAs+tcKJPHAdkpsnv37gCKt/bm0aNHdSzLDK+55prLPk+WC5E3X2vuydKu6dOnm0wn7PTt\n21fH99xzD4D8sjfAsxslXSS7ysr9t3///jpW+7AqTQZ8rzX94osv6jg2NlbHsuu+eg11fA51qlxz\nwYIFeuzDDz/UcenS+acEderUAeC7xD4Q6lYT+ft79tlndfzSSy9Zuj2nefLJJwEUrfRYdVM2sdZ1\nYXhlk4iIiIiIiCzHk00iIiIiIiKyXNiW0apyLgD4xz/+oeMTJ07o+IUXXjCaU0lVWOeyESNG6Jhd\naM2pV6+e19ixY8dsyCS8LV++XMfXX399kZ+3bds2Ha9bt87SnOy2Y8cOHcsOkgkJCQCARo0aFfm1\nZNdJadasWToeMGCA1+OyOy5dVLt2bR3L0kNFLrS+adMmIzmFq9tuu81rbOnSpTr+7rvvTKYTcmRJ\nrYyLSh4fZFmpLKO95ZZbAABVqlTRY6ozbihS3Uflv+2YmBifP3vrrbcC8Fx5YMyYMTou6LaqopK3\nV7Rs2TKg13K6oUOH6liVHMuSZykzM1PHsuOy3Xhlk4iIiIiIiCwXdlc21dpiEyZM0GMRERE6llcp\nNmzYYC6xECa/9SvqOoHHjx/3+Rz5LZqvtbMqVaqk48KuuMo1hNQ6W7m5uUXKLxT06NHDa+zzzz+3\nIZPQIL9J9dX0wNeVBgCYMWOGjmvWrOn1uHyt4qztaGLdz5JGrcOr/g7E7t27L/u4XJ9z69atAW/P\nCdq1a6djX/8GZLM3Ci55vDl9+jQA4M0337QrnbC2cOFCHcsrm3fddRcAz+qtcKl4+/LLL73GVGUK\n4Hll88KFCwCA9957T4/NnDlTx4888oiOfVVUkDe5dqY8LkRFRXn9rKwoVE2BAODs2bNByq74eGWT\niIiIiIiILMeTTSIiIiIiIrJcWJTRyjJZtXZm/fr19diuXbt0LJsFUdH88MMPxX7ORx99pOMDBw7o\nuFq1ajpWJSxWOHjwIADg5Zdftuw17ZCUlKTj6tWr25hJ6Jk6daqOX3/9da/HZXOOgsphCyuTLezx\nadOmXfZxKjpZFi1jhaWz3tRtJJdSa5mmpqaaTCfsyBI3+V53+PBhAGwKZBd53JbvDb179wbguW7w\n/PnzdZyVlWUgu5Ljiy++0LH8LKWa1QwbNkyPySZwN99882VfVzYmo4vkbTZXXXWV1+Oq9B7wLP1O\nT08PbmJ+4pVNIiIiIiIishxPNomIiIiIiMhyYVFG27BhQx37Ws9HdjWVJbV0kerQq0pKrNC3b98i\n/6zqdAb4LlNcsmSJjgtaG27t2rXFyK7kSk5O1rEqD8/IyNBj33zzjfGcQoVcc+qJJ57QcXR0tGXb\nOHLkiI63b98OALj//vv1mCwZp8C43W6fMRWsS5cuPsezs7MBeHYJJ+vJMlq5zy5btszrZ2XpXOXK\nlXWsflcUHLJT9nPPPQcAGDdunB4bO3asjgcOHAggfNb0Ve9pgGcHX7mWsqLWKL2UWiVA7vNPPfWU\nVSmGNPlv/sknn7zsz86dO1fHaWlpwUrJMryySURERERERJbjySYRERERERFZzrFltPXq1dOx7KCl\nyDI62YWSvN1xxx0APC/rlylT5rLPadq0qY4L6yr77rvv6njPnj1ej3/yySc63rFjx2Vfy4muvPJK\nHXfr1s3r8Y8//ljHqkSFvO3du1fHKSkpOr799tsBACNHjgx4G7JD3+TJkwN+PSpYuXLlvMbCpZyt\nOOSxWt5SIv3+++8AgPPnzxvJiTyp4/aAAQP02KhRo3ScmZmp43vvvddcYmFu9uzZAIDhw4frMfV5\nCABeeOEFAP515A9F8vj6yCOP6DgqKgoAcMMNN+ixqlWr6lh+rpszZw4AYMyYMUHKMvSo+du2bZse\nK+gzttrX5PyHAl7ZJCIiIiIiIsu5gt1YweVy2dK5QV5hePrpp70eb926tY4LaipjF7fb7b1oXBHY\nNdehyt95BszOtfyG6+uvv9axWputf//+eiw3N9dUWsUSCvt0165ddSyb+sj1rlQzqhkzZugxucaj\n/GbSrkYeoTDXVlBr5wL567y9+OKLeszEmpGhMNdynel//etfOr7vvvt0rK7glOSrZqEw14WRzWfi\n4+N1rI4h8vPYO++8o2O5X+/bty+YKYbM+6JJdevW1bG8Sjdv3jwAnleki8MJ+7SkGiYBQNu2bXX8\n/PPP61h9bjGtJM+1Widz8eLFeqygc7Nbb70VALBmzZpgp+U3X3PNK5tERERERERkOZ5sEhERERER\nkeUcVUablJSkY7U2JJB/863EMlpiuZA53KfNCZe5/vzzz3U8fvx4AOZLi0JtrmvWrKnjl156Sceb\nN28GULKbWoXaXPsiP6Oo5jJA/vrIU6dO1WPHjh3T8blz5wxkdxHfFy9PNpy88cYbAQBt2rTRY/JW\nisI4YZ8OFSV5rr///nsAnqX1klzndfTo0cFOJ2AsoyUiIiIiIiIjeLJJRERERERElnPUOps33XST\njn2VzgLArl27AACnTp0ykhMREVlPdgmmovn55591PHjwYBszCU/r1q3TcYcOHWzMhPzVp08fHavy\nx0aNGumx4pTREgFAlSpVAHh2tpdde99++23jOVmNVzaJiIiIiIjIcjzZJCIiIiIiIss5qoy2IKrU\nAchfEPXXX3+1Kx0iIiIiCjEnTpzQcf369W3MhJxCdVNXfwPAiy++qOMDBw4Yz8lqvLJJRERERERE\nlnPUOptOUZLXA3ISridmDvdpczjX5nCuzeFcm8H3RXO4T5vDuTaH62wSERERERGRETzZJCIiIiIi\nIssFpUFQZGTkwd9//72aP88tV67coTNnzlS3Oien8neuOc/Fx7k2h3NtBo/V5nCuzeHxwxzOtTmc\nazN4rLZeUO7ZdLlcbn9f1+VyBXTPQLjxd645z8XHuTaHc20Gj9XmcK7N4fHDHM61OZxrM3isth7L\naImIiIiIiMhyPNkkIiIiIiIiy/Fkk4iIiIiIiCxn7GQz65csJE5PRIvpLZA4PREVX62ICf+ZYGrz\nYSXnRA46zOqAplOaIn5qPOc5iIYsHoJqb1RDs6nN7E7F8VbuXInGkxojZmIMXlv3mt3pOF6eOw8t\nprdAr3m97E7FsXj8MIv7tBmpG1IRPzWenz8MuPbta9F8WnMkTk9E65mt7U7H0bhf+8/YyWbM1THI\nGJ6B74Z/h833b0b5MuWR3DjZ1ObDSulSpTG+y3hkPpCJ9UPWY/L/m4wdR3fYnZYjDUochFV3r7I7\nDcfLc+dhxPIRWHX3KmQ+kIl5W+dxnw6y1A2paBLdxO40HI3HD7O4Twdf5uFMvJPxDjYN24Qtw7dg\nadZS7D622+60HKuUqxTS7k1DxvAMbBy20e50HIv7dWBsKaNdvXs1GlZpiDoV69ixecerHlUdCdUT\nAABRV0Qh9ppY7D+x3+asnCmpbhIql6tsdxqOt3H/Rlx39XWoV6keykSUQUpcChbvWGx3Wo6VcyIH\ny3cux9AWQ+1OxdF4/DCH+7QZ249uR5tabVC2dFlElIpA+3rtsWj7IrvTciw33Mhz59mdhuNxvw6M\nLSebC7YSi1lCAAADOElEQVQuQL+4fnZsOuzs+W0Pthzcgja129idCpHf9p/YjzoV8r+cql2hNvaf\n5BcowTJq1SiM6zQOLrCDOzkD92kz4qrGYW32Whw7cwy553Ox/P8vx77j++xOy7FccKHTnE5oNbMV\nZm6eaXc6jsX9OjClTW/w/B/nsSRrCV7t+KrpTYedU+dOoc/CPkjtmoqoK6LsToeIQsCyrGWoVr4a\nEqonIG1PGtywfi1mIpO4T5vT+JrGGP2n0eg0pxOirohCYvVERJSKsDstx0ofnI4aV9XAkdNH0GlO\nJ8RGxyKpbpLdaTkO9+vAGL+yuWLnCrSs0RLR5aNNbzqsXMi7gD4L+2Bgs4Ho3bi33ekQBaRWhVrI\nPp6t/zvnRA5qXVXLxoycK31fOpb8uAQNUhug3yf9sOanNbjn03vsTovIb9ynzRqUOAib7t+EtPvS\nUKlcJcRcHWN3So5V46oaAIDo8tFIbpyMjft532awcL/2n/GTzXlb57GE1oDBiwejSXQTjGw70u5U\nHM/9v/9R8LSq2Qo7f92Jvb/txbk/zmH+1vnodT07SgbD2FvHIntUNnaP3I35d85Hh/odMDt5tt1p\nORaPH8HHfdqsI6ePAACyj2fj0x2fon98f5szcqbc87k4de4UAOD0udP4YvcXiKsaZ3NWzsX92n9G\ny2hzz+di9e7VmNFjhsnNhp307HTM/e9cxFeNR+L0RLjgwthbx6Jro652p+Y4/T/pj7Q9afjlzC+o\n+1ZdPH/z8xiUOMjutBwnolQEJnWbhM4fdEaeOw9DEocgNjrW7rSIAsLjBznRnQvvxK9nfkWZiDKY\n0n0KKpStYHdKjnTo1CEkL0iGy+XChbwLGBA/AJ0bdrY7Lcfifu0/l9tt/TeqLpfL7e/rulwuuN1u\n3sFfRP7ONee5+DjX5nCuzeCx2hzOtTk8fpjDuTaHc20Gj9XWs6UbLRERERERETkbTzaJiIiIiIjI\ncjzZJCIiIiIiIsvxZJOIiIiIiIgsF5RutOXKlTvkcrmq+ftcq/NxMn/nmvNcfJxrczjXZvBYbQ7n\n2hweP8zhXJvDuTaDx2rrBaUbLREREREREYU3ltESERERERGR5f4PQ+8P2ZkutsAAAAAASUVORK5C\nYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x27815670>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "def make_image(X):\n",
    "    im = np.swapaxes(X.T, 0, 1)\n",
    "    im = im - im.min()\n",
    "    im = im * 1.0 / im.max()\n",
    "    return im\n",
    "\n",
    "plt.figure(figsize=(16, 5))\n",
    "for i in range(0, 10):\n",
    "    plt.subplot(1, 10, i+1)\n",
    "    plt.imshow(make_image(test_data[i][0]), interpolation='nearest', cmap=plt.get_cmap('gray'))\n",
    "    true = test_label[0][i]\n",
    "    pred = FPGA_predicted[i]\n",
    "    color = 'green' if true == pred else 'red'\n",
    "    plt.text(0, 0, true, color='black', bbox=dict(facecolor='white', alpha=1))\n",
    "    plt.text(0, 32, pred, color=color, bbox=dict(facecolor='white', alpha=1))\n",
    "\n",
    "    plt.axis('off')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "### Save our model\n",
    "Let's save the weights in pickle format, so we don't need Caffe next time"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import pickle\n",
    "\n",
    "values = lasagne.layers.get_all_param_values(net['output'])\n",
    "pickle.dump(values, open('model.pkl', 'w'))"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.4.3+"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}
